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SEGMENTATION  OF  SYNTHETIC  APERTURE  RADAR  (SAR) 
IMAGES  OF  OCEAN  SURFACE  BY  THE  TEXTURE  ENERGY 
TRANSFORM  METHOD 


I  ntroduct I  on 


Texture  analysis  is  an  Important  aspect  of  image  c l ass i f ' '■'* r  i  on 
and  segmentation  tasks.  it  is  fundamental  to  some  app l icat ions  such 
as  metal  surface  analysis,  radiographic  diagnosis,  identification  of 
different  terrains  In  aerial  photographs,  isolating  special  regions 
In  natural  scenes,  etc..  The  subject  has  been  studied  quite 
extensively  [ 1 ] .  Different  statistical  approaches  have  been  employed 
to  evaluate  quantitative  feature  measures  from  Image  pixel  data  for 
texture  characterization.  Despite  the  ingenious  ideas  used  and  the 
success  accomplished  in  classifying  the  images  chosen  in  the  previous 
researches,  most  texture  analysis  approaches  st i i i  have  limitations. 
They  were  applied  to  a  limited  categories  of  images  and  may  not  work 
well  In  general.  All  of  them  require  ted  I  us  time-consuming  machine 
process i ng . 

Recently,  Laws  [2]  Introduced  a  different  kind  of  approach  to 
texture  measures  which  he  cal  led  the  texture  energy  transform.  in 
this  approach,  texture  features  are  evaluated  in  a  spatial- 
statistical  sequence  of  small  mask  convolutions  followed  by  moving 
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window  averaging.  The  results  are  closely  related  to  the  degree  of 
match  between  the  pixel  ne l ghbour hoods  and  the  set  of  chosen  masks. 

In  addition  to  possessing  an  Intuitive  physical  reasoning  in  the 
formulation,  these  features  are  also  simpler  to  compute.  Application 
of  this  approach  to  texture  classification  yielded  better  results 
than  other  standard  techniques  based  on  second  order  gray  level 
statistics  [2,3].  it  can  also  be  made  invariant  to  changes  in 
luminance,  contrast,  and  rotation  without  the  use  of  histogram 
equalization  or  other  preprocess i ng . 

In  this  report,  we  present  the  results  of  texture  segmentation 
of  SAR  Images  of  ocean  surface  using  this  new  technique.  It  is  a 
continuation  of  the  task  reported  previously  where  similar  images 
were  classified  by  the  gray-tone  co-occur rence  matrix  approach  [4], 
The  previous  results  obtained  are  not  as  good  as  we  had  hoped  for 
which  prompted  the  search  for  a  different  processing  method.  Our 
objective  of  mapping  the  image  into  two  kinds  of  regions 
differentiated  by  the  presence  and  absence  of  pixel  streak  patterns 
has  been  successful.  Physically  speaking,  this  separation  process 
Identifies  the  areas  of  ocean  surface  where  significant  difference 
existed  In  terms  of  surface  wind  patterns  and  wave  structures. 

Texture  Energy  Transform  Approach 

The  processing  sequence  of  the  texture  energy  transform  approach 
to  image  segmentation  is  outlined  In  Figure  1.  The  operation  steos 
involved  are  shown  in  the  upper  part  of  the  figure  with  the 
associated  frames  of  data  attached  below.  a  more  detai  led 
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(a)  Operation  sequence 


(b)  image  frame  sequence 
Fig.  1.  Texture  energy  transform  approach. 

The  original  I  mage ,1,  Is  first  filtered  by  a  set  of  micro-filter 
convolution  masks  with  Integer  elements  to  generate  the  set  of  F 
plane  Images.  The  pixel  data  In  an  F  plane  image  is  a  function  of 
the  original  image  and  the  chosen  fi  iter.  They  can  assume  negative 
as  we  li  as  po3  fttve  values.  These  filtered  images  are  o  i  ane  -  ames 
of  data  in  the  processing  sequence  and  may  not  represent  recognizable 
images.  The  f i Itered  F  plane  images  are  then  processed  pv 
evaluating  statistics  of  data  within  a  scanning  macro-window  jna 


assigning  the  result  to  the  center  pixel  to  produce  the  £  plane 


images.  This  Is  a  continuous  reg t on-to-po l nt  transf ornat I  on 


where  a  pixel  In  a  E  plane  image  acquires  a  number  descriptive  of  its 
Immediate  neighbourhood  In  the  original  image.  The  size  of  the 
convolution  mask  Is  chosen  to  be  comparable  to  that  of  the  typical 
texture  primitives  of  the  original  Image.  The  macro-window  should 
have  a  size  appropriate  to  cover  one  kind  of  texture  region  most  of 
the  time  and  two  kinds  of  regions  along  the  boundary  of  region 
separation.  It  Is  chosen  so  that  the  statistics  evaluated  will 
reveal  the  difference  when  it  is  located  within  different  regions. 
These  statistics  are  more  representative  of  local  spatial  variations 
than  typical  frequency  domain  measures  and  are  suitable  for  textures 
with  short  coherence  length  or  correlation  distance.  To  reduce  the 
required  processing  time,  the  dimensions  of  masks  ana  windows  are 
chosen  sufficiently  small  to  be  physically  reasonable  for  the  images 
under  study. 


if  there  are  many  E  plane  Images  chosen,  an  optional  step  may  be 


taken  to  compress  them  into  a  sma i l  number  of  principal  component  C 
plane  Images  wnicfi  still  carry  enough  discriminating  information. 

This  can  be  achieved  by  using  the  Hotelling  transform  which  picks  out 
the  orthogonal  Image  components  with  the  largest  variance  [5]. 

The  texture  energy  plane  images  or  the  principal  component  plane 
images  are  the  Input  to  the  classifier  which  assign  each  pixel  the 
proper  texture  label  to  produce  the  final  segmented  Image  M. 
Classification  is  simple  if  texture  classes  are  known  and  training 
samples  are  available.  Clustering  and  segmentation  algorithms  must 
be  used  in  addition  to  that  of  the  classification  algorithm  if 
texture  classes  are  unknown  and  have  to  be  Identified. 

Experimental  Methods 


In  the  first  step  of  the  texture  processing,  the  original  image 
Is  filtered  by  a  set  of  convolution  masks.  These  masks  are 
constructed  as  two  d imens lonal  matr l x  mu itlplicatlon  of  the  linear 
masks  shown  In  Figure  3.  These  linear  masks  are  called  Lattice 
Aperture  Waveform  Sets  of  order  three,  five  and  seven.  The  names  of 
the  vectors  are  mnemonics  for  level,  edge,  spot,  wave,  ripple, 
undulation,  and  oscillation.  All  of  them  are  symmetrically  weighted 
toward  their  center.  Except  for  the  level  vectors,  tne  sum  of  a l i 
the  elements  in  each  eauates  to  zero.  Linear  vectors  of  ■ ength  three 
form  the  basis  since  the  longer  ones  can  be  derived  in  terms  of  these 
three  through  repeated  l  I  near  convolution. 


Figure  4  shows  the  nine  square  f i  1  ter  masks  generated  oy  the 


three  l Inear  vectors  of  length  three.  4ny  3  by  3  mask  can  be 
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Fig.  3.  Center -we l ghted  mask  filters. 

expressed  as  a  unique  linear  combination  of  them.  The  names  of  these 
masks  attached  underneath  are  coined  by  the  names  of  the  two  i inear 
vectors  constituting  the  column  and  row  vectors  of  the  cor  respond i ng 
matrix  of  the  mask.  5  by  5  and  7  by  7  as  we i i  as  non-sauare 
rectangular  masks  can  be  formed  In  a  simi lar  fashion.  Figure  5  shows 
four  5  by  5  masks  which  were  found  to  be  most  useful  in  the 
experimental  work  done  by  Laws  [2].  The  structure  of  these  masks  as 
the  matrix  product  of  the  elements  :  r.  L.-.e  basis  set  helps  to  simpiicv 
the  algorithm  and  reduce  the  computation  time  in  the  step  of 


convolution  filter  processing. 
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Fig.  4.  3  by  3  center-weighted  mask  filters. 


Each  mask  filter  has  a  unique  combination  of  elements. 

Con vo lutlon  wilt  produce  a  plane  of  positive  or  negative  numbers  or 
larger  magnitude  If  the  spatial  pixel  data  variation  of  the  ongina 
image, l,  matches  closely  that  of  tne  mask.  A  homogeneous  image 
region  which  shows  no  textural  structure,  or  a  textural  region  wncse 
spatial  structure  differs  f rom  that  of  the  mask  will  result  in 
smaller  numbers.  The  data  In  the  F  plane  image  reveals  tne  degree  o 
texturai  similarity  between  the  masx  and  the  corr espond i ng  i mmed late 


vicinity  In  image  i . 


Pixels  located  within  different  textural 


primitives  in  linage  l  will  have  numerical  values  of  different  levels 


in  plane.  Hence,  the  transformation  from  l  to  F  provides  a 
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Fig.  5.  5  t>y  5  center-we  1  ghted  mask  filters. 

separation  step  among  different  texture  primitives.  The  pattern  and 
the  steepness  in  separation  among  the  texture  primitives  vary  among 
the  different  masks. 

F am i  liar  statistics  sucn  as  mean,  variance,  standard  deviation, 
range,  max. mum,  mlnimun,  sxewness,  Kurtosis,  and  etc.  can  oe  chosen 
as  the  texture  feature  representation  in  the  macro-window  processing 
of  F  plane  .mages  to  generate  E  plane  images.  Laws  concluded  n  n«s 
study  that  variance  or  standard  deviation  alone  is  sufficient  in 
extracting  the  necessary  Information.  if  all  mask  filters  chosen  , n 

■< 


the  convolution  process  have  zero  sum  elements,  the  F  plane  image  is 


essentially  a  zero  mean  pixel  field.  Variance  is  an  average  of 
squared  deviation  from  the  mean.  with  a  vanishing  mean  value,  tne 
variance  becomes  the  average  of  the  squared  pixel  values  within  the 
macro-window.  In  this  sense,  each  pixels  value  in  E  plane  image  is 
an  energy  measure  of  Its  macro-window  ne I ghbour hood  In  the  F  plane. 

To  simplify  the  calculation  and  still  maintain  esentialiy  the  same 
statistical  meaning,  the  average  of  the  absolute  values  within  the 
macro-window  was  chosen  as  the  statistic  and  its  value  is  assigned  to 
the  center  pixel  of  the  window  in  the  E  plane  image. 

For  each  selection  of  the  mask  and  statistic  combination,  there 
Is  one  feature  measure  of  the  texture.  Since  the  choice  of  the 
statistic  Is  a  fixed  one,  the  number  of  masks  chosen  determines  the 
number  of  features  to  be  computed  and  the  number  of  E  plane  images  to 
be  generated.  The  discriminating  power  of  a  feature  is  Judged  by  the 
F-ratlo  statistic  calculated  from  the  available  training  sample  image 
data  by  the  fol lowing  formula  from  [0], 

F-(varlance  of  inter-class  sample  means )/( comb i ned 
Intra-class  variance) 

A  good  choice  of  feature  w l i l  generate  values  witn  smal i  variance 
from  texture  samples  of  the  same  type  and  significantly  different 
mean  values  of  samples  of  different  types.  Better  choices  of 
features  will  produce  higher  F-ratio  statistics.  The  statistics, 
meaning  of  the  F-ratlo  also  depends  upon  the  number  of  the  samples 
involved.  The  comparison  of  these  ratios  has  to  oe  made  on  equal 


number  of  samples  to  yield  accurate  Judgement. 


With  many  choices  of 


features  available  for  characterization  of  differences  among  the 
textural  categories,  the  proper  combination  of  them  to  generate  tne 
best  accuracy  of  dlscrlmatlon  can  be  obtained  through  discriminant 
analysis  under  certain  statistical  criteria  [2,7].  in  the  procedure 
of  segmenting  an  Image,  the  feature  vector  composed  of  chosen  feature 
elements  evaluated  for  each  pixel's  ne l ghbour hood  is  compared  with 
the  feature  vectors  evaluated  by  the  same  formulas  from  known 
training  samples  of  each  class.  A  pixel  Is  classified  as  belonging 
to  the  class  whose  centroid  vector  has  the  least  distance  separation 
from  the  feature  vector  evaluated  in  its  vicinity. 

Experimental  Results  and  Olscusslons 


Figures  6a, 7a, and  8a  show  the  Images  which  we  wish  to  segment 
using  the  texture  energy  transform  method.  These  are  sub-images 
obtained  during  the  SlR-8  mission  on  October  11,  1984.  Each  one  has 

a  size  of  512  by  512  pixels.  The  area  covered  is  outside  the  US. 
east  coast  with  the  center  geographic  location  around  37  degree  north 
and  74  degree  west.  By  carefully  examining  these  images,  we  can 
realize  that  there  are  two  kinds  of  textures  present.  First,  the 
pixel  variations  Indicate  directional  streak  patterns.  in  other  k i nc 
of  texture,  the  pixel  variation  seems  homogeneously  random.  The 
appearance  of  these  two  textures  indicates  the  difference  in  the 
radar  back-scattering  character  I st ics  from  different  areas  of  ocean 
surface.  Our  Image  segmentation  objective  is  to  separate  these  two 
textural  areas  where  contrasting  surface  meteorological  conditions 
existed  when  the  Image  was  recorded. 


:o 


According  to  Laws'  research,  5  by  5  masks  were  more  powerful 
than  the  3  by  3  masks,  and  much  simpler  to  process  than  7  by  7  masks. 
By  detailed  comparison,  we  found  that  the -5  by  5  mask  Is  big  enough 
to  cover  an  essential  part  of  a  typical  texture  primitive  in  these 
Images.  We  chose  to  try  the  masks  shown  In  Figure  5  first.  Four 
training  samples  with  size  of  64  by  64  pixels  of  streak  areas  and 
non-streak  areas  were  located  in  each  image. 

Since  there  are  only  two  classes  of  texture  to  be  separated,  the 
complicated  discriminant  analysis  is  not  necessary  In  this  problem. 
Each  five  by  five  mask  was  convolved  with  each  of  the 
training  samples  and  the  average  of  absolute  values  of  the 
convolution  was  then  computed.  There  are  small  differences  between 
these  two  kinds  of  textures  and  the  results  are  not  much  better  than 
what  had  been  achieved  using  the  co-occurrence  matrix  method  [4], 
However,  mask  filters  E5L5  and  L5S5  did  produce  consistently  more 
difference  than  the  other  two  filters.  E5L5  is  a  horizontal  edge 
mask  and  L5S5  Is  a  vertical  line  detector.  This  fact  indicates  that 
the  edge  and  line  detector  masks  can  separate  image  regions  with 
streaks  and  without  streaks  more  effectively  than  masks  wh.ch 
discover  textural  detail  In  other  aspects.  Attention  was  shifted 
toward  finding  the  best  possible  l ine  detector  masks  which  are  zero- 
sum  and  made  of  simple  Integer  components.  Judgement  »as  made  on 
tests  done  with  the  training  samples  of  each  image.  The  most 
powerful  masks  found  this  way  are  shown  in  Figures  9.  ’0.  and  i!  £cr 

the  'mages  n  J ■ gures  6a.  7a,  and  3a  respectively  "he  two  masks 
shown  in  each  figure  are  a  90  degree  transpose  of  each  other.  One  of 
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them  detects  the  vertical  I l nes  and  edges  and  the  other  detects  the 
horizontal  counter car ts .  Both  were  utilized  simultaneously  as  the 
direction  of  the  streaks  can  De  random.  Applying  these  pairs  of 
filters  to  the  cor r espond l ng  training  samples  and  taking  the  sum  of 
the  two  average  absolute  values  in  each  case  yields  two-ooint  scatte 
plots.  Two  threshold  levels  for  each  image  to  separate  the  streak 
and  non-streak  textures  as  we  I  I  as  a  transition  region  were 


determined.  Static  threshold  levels  chosen  this  way  were 
incorporated  into  the  process  to  classify  the  corresponding  whole 

image. 

Starting  at  the  upper  left  corner,  the  mask  filters  In  Figures 
9,  10,  and  11  were  scanned  pixel  by  pixel  to  compute  convolution 

through  the  whole  Image.  A  moving  macro-window  of  size  33  by  33 
pixels  was  chosen  so  that  there  Is  a  precisely  defined  center 
location.  The  macro-window  is  also  scanned  pixel  by  pixel  through 
the  image.  At  each  position  In  the  scanning  process,  the  sum  of  the 
average  absolute  values  of  the  convolutional  results  of  the  two 
perpendicular  masks  within  the  window  was  taken.  This  sum  Is  then 
adjusted  to  eliminate  the  influence  due  to  the  difference  In  the  mean 
pixel  value  within  the  window  at  each  position  in  the  original  image 
This  adjusted  sum  Is  compared  with  the  two  threshold  levels  to 
determine  the  center  pixel's  classification.  Segmentation  results 
are  shown  In  Figures  60,  7b,  and  8b  for  the  Images  in  Figures  6a,  7a, 
and  8a  respectively.  Pseudo  colors  are  used  for  clarification. 

Green  color  Is  assigned  to  the  streak  textural  regions  and  the  blue 
color  Is  assigned  to  the  non-streak  textural  regions.  The  transition 
regions  are  cyan.  Sixteen  pixels  around  the  edge  could  not  be 
classified  and  are  colored  red.  The  threshold  levels  are  adjustable. 
The  threshold  level  chosen  for  streaked  regions  of  Figure  6  was  more 
stringent.  As  a  conseauence,  less  area  was  classified  as  streaked 
region  In  Figure  6  as  compared  to  those  in  the  other  two  images.  The 
lower  left  corner  of  Figure  6  was  not  accurately  classified  because 
the  original  pixel  data  there  was  Incomplete.  The  size  of  the  macro¬ 
window  can  be  varied,  but  Its  optimum  size  may  not  be  easy  to  find. 


;o 


in  some  parts  of  these  Images  the  perception  does  change  somewhat  as 
the  window  size  varies  or  as  the  average  pixel  Intensity  within  the 
window  varies.  However,  careful  scrutiny  of  tne  corresponding 
images  shows  that  these  results  do  present  accurate  maps  which 
separate  the  two  kinds  of  textures. 

Cone i us l ons 

Tfie  texture  energy  transform  Is  the  latest  method  developed  for 
texture  analysis  of  images.  it  is  based  on  a  spat  I  a  I -stat I st i ca i 
approach  In  which  texture  features  are  determined  by  the  degree  of 
match  between  pixel  ne I ghbour hood  of  the  Image  and  a  set  of  chosen 
mask  f l Iters.  The  method  Is  conceptual ly  simple  and  consumes  less 
computation  time  than  most  other  established  texture  classification 
techniques.  It  was  app l led  to  segmentation  of  synthetic  aperture 
radar(SAR)  Images  of  ocean  surface  Into  regions  with  and  without 
directional  streaks.  Excellent  results  were  obtained.  However,  the 
set  of  masks  Incorporated  In  the  analysis,  which  worked  we  I  l  ,  are 
masks  detecting  l l nes  ana  edges  rather  than  the  standard  set 
originally  proposed  to  discover  essentially  all  possible  spatial 
variations.  This  fact  Indicates  that  the  textural  difference  between 
the  regions  with  and  without  streaks  are  subtler  than  we  anticipated 
The  distinction  can  be  recognized  without  much  difficulty  with  naked 
eyes  but  It  may  not  be  easy  to  establish  by  algorithms  based  on 
various  conventional  texture  discriminating  methods.  Since  both  the 
co-occurrence  matrix  method  and  the  texture  energy  transform  method 
produced  the  same  level  of  difference  in  feature  measures  between  tne 
two  kinds  of  the  training  samples,  we  be  I  I  eve  they  are  almost  eoua  y 


powerful  in  texture  analysis.  For  the  SAR  ocean  surface  Images  we 
tried  to  segment,  It  appears  more  efficient  to  search  for  the 
.existence  of  lines  or  edges  rather  than  for  two  different  Kinds  of 
textures.  it  Is  unlikely  that  any  other  powerful  texture  analysis 
method  will  be  able  to  discern  these  two  regions  In  terms  of  clear 
cut  texture  dissimilarity. 

The  size  of  the  micro-filter  masks  and  the  macro-winaow  for 
texture  energy  evaluation  are  determined  individual ly  for  each  task 
under  Investigation  to  match  the  intrinsic  spatial  texture  variations 
and  each  textural  region's  typical  extent  of  coverage.  The  masks 
suggested  by  Laws  [2]  cover  a  very  general  case  in  which  many  kinds 
of  drastically  different  textures  are  Involved.  Laws  masks  are 
derived  by  a  two  dimensional  matrix  mu l t i p l  l cat  ion  of  two  I  Inear 
vectors  which  are  obtained  from  sequence  of  convolutions  with  three 
vectors  of  length  three.  Choice  of  these  masks  can  simplify  the  mask 
convolution  calculation  and  save  processing  time.  The  optimum 
combination  of  masks  which  make  the  set  for  the  feature  vector 
components  can  be  determined  through  F-ratio  statistic  test  and 
discriminant  analysis  with  the  training  samples.  For  simpler 
problems  with  fewer  classes  of  textures  to  segment,  I  Ike  the  one 
studied  in  this  report,  one  or  two  feature  measures  which  yield 
clearly  different  range  of  values  among  different  classes  may  be 
enough  and  the  F-ratio  test  Is  not  necessary.  Proper  workable  masks 
can  be  found  by  trial  and  error  and  the  textural  identification  can 
be  done  with  threshold  level  separation  instead  of  the  comp i  catec 
statistical  classifications. 
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